Apparatus and methods for vector operations

ABSTRACT

Aspects for vector operations in neural network are described herein. The aspects may include a vector caching unit configured to store a first vector and a second vector, wherein the first vector includes one or more first elements and the second vector includes one or more second elements. The aspects may further include one or more adders and a combiner. The one or more adders may be configured to respectively add each of the first elements to a corresponding one of the second elements to generate one or more addition results. The combiner may be configured to combine a combiner configured to combine the one or more addition results into an output vector.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention is a continuation-in-part of PCT Application No.PCT/CN2016/081107, filed on May 5, 2016, which claims priority tocommonly owned CN application number 201610266989.X, filed on Apr. 26,2016. The entire contents of each of the aforementioned applications areincorporated herein by reference.

BACKGROUND

Multilayer neural networks (MNN) are widely applied to the fields suchas pattern recognition, image processing, functional approximation andoptimal computation. In recent years, due to the higher recognitionaccuracy and better parallelizability, multilayer artificial neuralnetworks have received increasing attention by academic and industrialcommunities. More specifically, operations between two vectors may beperformed frequently in deep learning processes in MMNs.

A known method to perform operations for two vectors in a multilayerartificial neural network is to use a general-purpose processor.However, one of the defects of the method is low performance of a singlegeneral-purpose processor which cannot meet performance requirements forusual multilayer neural network operations with respect to a vector witha large number of elements.

Another known method to perform operations for two vectors of themultilayer artificial neural network is to use a graphics processingunit (GPU). Such a method uses a general-purpose register file and ageneral-purpose stream processing unit to execute general purposesingle-instruction-multiple-data (SIMD) instructions to support thealgorithms in MNNs. However, since GPU only contains rather smallon-chip caching, then data of the vector elements may be repeatedlymoved from the off-chip, and off-chip bandwidth becomes a mainperformance bottleneck, causing huge power consumption.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

One example aspect of the present disclosure provides an exampleapparatus for vector operations in a neural network. The exampleapparatus may include a vector caching unit configured to store a firstvector and a second vector, wherein the first vector includes one ormore first elements and the second vector includes one or more secondelements. Further, the example apparatus may include a computationmodule that includes one or more adders and a combiner. The one or moreadders may be configured to respectively add each of the first elementsto a corresponding one of the second elements to generate one or moreaddition results. The combiner may be configured to combine the one ormore addition results into an output vector.

Another example apparatus may include a vector caching unit configuredto store a first vector and a second vector, wherein the first vectorincludes one or more first elements and the second vector includes oneor more second elements. The example apparatus may further include acomputation module that includes one or more multipliers and a combiner.The one or more multipliers may be configured to respectively multiplyeach of the first elements with a corresponding one of the secondelements to generate one or more multiplication results. The combinermay be configured to combine multiplication results into an outputvector.

Another example aspect of the present disclosure provides an examplemethod for vector operations in a neural network. The example method mayinclude storing, by a vector caching unit, a first vector and a secondvector, wherein the first vector includes one or more first elements andthe second vector includes one or more second elements; respectivelyadding, by one or more adders of a computation module, each of the firstelements to a corresponding one of the second elements to generate oneor more addition results, and combining, by a combiner of thecomputation module, the one or more addition results into an outputvector.

The example aspect of the present disclosure may include another examplemethod for vector operations in a neural network. The example method mayinclude storing, by a vector caching unit, a first vector and a secondvector, wherein the first vector includes one or more first elements andthe second vector includes one or more second elements; respectivelymultiplying, by one or more multiplier of a computation module, each ofthe first elements with a corresponding one of the second elements togenerate one or more multiplication results; and combining, by acombiner, the one or more multiplication results into an output vector.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features herein after fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed, and this description is intended to include all suchaspects and their equivalents.

BRIEF DESCIPTIOIN OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements, andin which:

FIG. 1 illustrates a block diagram of an example neural networkacceleration processor by which vector operations may be implemented ina neural network;

FIG. 2A illustrates an example vector addition process that may beperformed by the example neural network acceleration processor;

FIG. 2B illustrates an example vector multiplication process that may beperformed by the example neural network acceleration processor;

FIG. 3 illustrates an example computation module in the example neuralnetwork acceleration processor by which vector operations may beimplemented in a neural network;

FIG. 4A illustrates a flow chart of an example method for performingvector addition between two vectors in a neural network;

FIG. 4B illustrates a flow chart of an example method for performingvector addition between a vector and a scalar value;

FIG. 5A illustrates a flow chart of an example method for performingvector subtraction between two vectors in a neural network; and

FIG. 5B illustrates a flow chart of an example method for performingvector subtraction between a vector and a scalar value in a neuralnetwork.

DETAILED DESCRIPTION

Various aspects are now described with reference to the drawings. In thefollowing description, for purpose of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofone or more aspects. It may be evident, however, that such aspect(s) maybe practiced without these specific details.

In the present disclosure, the term “comprising” and “including” as wellas their derivatives mean to contain rather than limit; the term “or”,which is also inclusive, means and/or.

In this specification, the following various embodiments used toillustrate principles of the present disclosure are only forillustrative purpose, and thus should not be understood as limiting thescope of the present disclosure by any means. The following descriptiontaken in conjunction with the accompanying drawings is to facilitate athorough understanding to the illustrative embodiments of the presentdisclosure defined by the claims and its equivalent. There are specificdetails in the following description to facilitate understanding.However, these details are only for illustrative purpose. Therefore,persons skilled in the art should understand that various alternationand modification may be made to the embodiments illustrated in thisdescription without going beyond the scope and spirit of the presentdisclosure. In addition, for clear and concise purpose, some knownfunctionality and structure are not described. Besides, identicalreference numbers refer to identical function and operation throughoutthe accompanying drawings.

Various types of vector operations between two vectors may be performedin a neural network. A vector may refer to one or more values formattedin a one-dimensional data structure. The values included in a vector maybe referred to as elements. The number of the elements in the vector maybe referred to as a length of the vector.

FIG. 1 illustrates a block diagram of an example neural networkacceleration processor by which vector operations may be implemented ina neural network.

As depicted, the example neural network acceleration processor 100 mayinclude a controller unit 106, a direct memory access unit 102, acomputation module 110, and a vector caching unit 112. Any of theabove-mentioned components or devices may be implemented by a hardwarecircuit (e.g., application specific integrated circuit (ASIC),Coarse-grained reconfigurable architectures (CGRAs), field-programmablegate arrays (FPGAs), analog circuits, memristor, etc.).

In some examples, a vector operation instruction may originate from aninstruction storage device 134 to the controller unit 106. Aninstruction obtaining module 132 may be configured to obtain a vectoroperation instruction from the instruction storage device 134 andtransmit the instruction to a decoding module 130.

The decoding module 130 may be configured to decode the instruction. Theinstruction may include one or more operation fields that indicateparameters for executing the instruction. The parameters may refer toidentification numbers of different registers (“register ID”hereinafter) in the instruction register 126. Thus, by modifying theparameters in the instruction register 126, the neural networkacceleration processor 100 may modify the instruction without receivingnew instructions. The decoded instruction may be transmitted by thedecoding module 130 to an instruction queue module 128. In some otherexamples, the one or more operation fields may store immediate valuessuch as addresses in the memory 101 and a scalar value, rather than theregister IDs.

The instruction queue module 128 may be configured to temporarily storethe received instruction and/or one or more previously receivedinstructions. Further, the instruction queue module 128 may beconfigured to retrieve information according to the register IDsincluded in the instruction from the instruction register 126.

For example, the instruction queue module 128 may be configured toretrieve information corresponding to operation fields in theinstruction from the instruction register 126. Information for theoperation fields in vector addition (VA) instruction may include astarting address of a first vector, a length of the first vector, astarting address of a second vector, a length of the second vector, andan address for an output vector. As depicted, in some examples, theinstruction register 126 may be implemented by one or more registersexternal to the controller unit 106.

Once the relevant values are retrieved, the instruction may be sent to adependency processing unit 124. The dependency processing unit 124 maybe configured to determine whether the instruction has a dependencyrelationship with the data of the previous instruction that is beingexecuted. This instruction may be stored in the storage queue module 122until it has no dependency relationship on the data with the previousinstruction that has not finished executing. If the dependencyrelationship does not exist, the controller unit 106 may be configuredto decode one of the instructions into micro-instructions forcontrolling operations of other modules including the direct memoryaccess unit 102 and the computation module 110.

For example, the control unit 106 may receive a vector additioninstruction that includes a vector addition (VA) instruction thatinclude a starting address of a first vector, a length of the firstvector, a starting address of a second vector, a length of the secondvector, and an address for an output vector. According to the VAinstruction, the direct memory access unit 102 may be configured toretrieve the first vector and the second vector from the memory 101according to the respective addresses included in the VA instruction.The retrieved vectors may be transmitted to and stored in the vectorcaching unit 112.

In some examples, the controller unit 106 may receive avector-add-scalar (VAS) instruction that includes a starting address ofa vector, a length of the vector, a scalar value, and an address for anoutput vector. According to the VAS instruction, the direct memoryaccess unit 102 may be configured to retrieve the vector from the memory101 according to the address. The vector may be transmitted to andstored in the vector caching unit 112. The scalar value included in theVAS instruction may be stored in the instruction caching unit 104. Theinstruction caching unit 104 may be implemented as a scratchpad memory,e.g., Dynamic random-access memory (DRAM), embedded DRAM (eDRAM),memristor, 3D-DRAM, non-volatile memory, etc.

In some examples, the controller unit 106 may receive avector-subtraction (VS) instruction that includes a starting address ofa first vector, a length of the first vector, a starting address of asecond vector, a length of the second vector, and an address for anoutput vector. According to the VS instruction, the direct memory accessunit 102 may be configured to retrieve the first vector and the secondvector from the memory 101 according to the respective addressesincluded in the VS instruction. The retrieved vectors may be transmittedto and stored in the vector caching unit 112.

In some examples, the controller unit 106 may receive ascalar-subtract-vector (SSV) instruction that include a starting addressof a vector, a length of the vector, a scalar value, and an address foran output vector. According to the SSV instruction, the direct memoryaccess unit 102 may be configured to retrieve the vector from the memory101 according to the address. The vector may be transmitted to andstored in the vector caching unit 112. The scalar value included in theSSV instruction may be stored in the instruction caching unit 104.

In some examples, the controller unit 106 may receive avector-multiply-vector (VMV) instruction that includes a startingaddress of a first vector, a length of the first vector, a startingaddress of a second vector, a length of the second vector, and anaddress for an output vector. According to the VMV instruction, thedirect memory access unit 102 may be configured to retrieve the firstvector and the second vector from the memory 101 according to therespective addresses included in the VMV instruction. The retrievedvectors may be transmitted to and stored in the vector caching unit 112.

In some examples, the controller unit 106 may receive avector-multiply-scalar (VMS) instruction that include a starting addressof a vector, a length of the vector, a scalar value, and an address foran output vector. According to the VMS instruction, the direct memoryaccess unit 102 may be configured to retrieve the vector from the memory101 according to the address. The vector may be transmitted to andstored in the vector caching unit 112. The scalar value included in theVMS instruction may be stored in the instruction caching unit 104.

In some examples, the controller unit 106 may receive a vector-divide(VD) instruction that includes a starting address of a first vector, alength of the first vector, a starting address of a second vector, alength of the second vector, and an address for an output vector.According to the VD instruction, the direct memory access unit 102 maybe configured to retrieve the first vector and the second vector fromthe memory 101 according to the respective addresses included in the VDinstruction. The retrieved vectors may be transmitted to and stored inthe vector caching unit 112.

In some examples, the controller unit 106 may receive ascalar-divide-vector (SDV) instruction that include a starting addressof a vector, a length of the vector, a scalar value, and an address foran output vector. According to the SDV instruction, the direct memoryaccess unit 102 may be configured to retrieve the vector from the memory101 according to the address. The vector may be transmitted to andstored in the vector caching unit 112. The scalar value included in theSDV instruction may be stored in the instruction caching unit 104.

The above mentioned instructions may be formatted as follows and may bestored in the instruction caching unit 104:

Register 0 Register 1 Register 2 Register 3 Register 4 VA StartingLength of Starting Length of Address for address the first address ofthe second output result of the first vector the second vector vectorvector VAS Starting Length of Address for Scalar value address thevector output result of a vector VS Starting Length of Starting Lengthof Address for address the first address of the second output result ofthe first vector the second vector vector vector SSV Starting Length ofAddress for Scalar value address the vector output result of a vectorVMV Starting Length of Starting Length of Address for address the firstaddress of the second output result of the first vector the secondvector vector vector VMS Starting Length of Address for Scalar valueaddress the vector output result of a vector VD Starting Length ofStarting Length of Address for address the first address of the secondoutput result of the first vector the second vector vector vector SDVStarting Length of Address for Scalar value address the vector outputresult of a vector

Hereinafter, a caching unit (e.g., the vector caching unit 112 etc.) mayrefer to an on-chip caching unit integrated in the neural networkacceleration processor 100, rather than other storage devices in memory101 or other external devices. In some examples, the on-chip cachingunit may be implemented as a register file, an on-chip buffer, anon-chip Static Random Access Memory (SRAM), or other types of on-chipstorage devices that may provide higher access speed than the externalmemory. In some other examples, the instruction register 126 may beimplemented as a scratchpad memory, e.g., Dynamic random-access memory(DRAM), embedded DRAM (eDRAM), memristor, 3D-DRAM, non-volatile memory,etc.

FIG. 2A illustrates an example vector addition process that may beperformed by the example neural network acceleration processor.

As depicted, a first vector (“Vector A”) may include one or moreelements respectively denoted as A(1), A(2), . . . A(n) and, similarly,a second vector (“Vector B”) may include one or more elementsrespectively denoted as B(1), B(2), . . . B(n). The elements in thefirst vector may be referred to as first elements. The elements in thesecond vector may be referred to as second elements.

The computation module 110 may include one or more adders. In responseto a VA instruction, each of the adders may be configured to add a firstelement in the first vector to a corresponding second element in thesecond vector e.g., A(1) to B(1), A(2) to B(2), . . . A(n) to B(n). Theaddition results generated respectively by the one or more adders may bedirectly transmitted to a combiner. In other words, the addition resultsmay be transmitted to the combiner without being temporarily stored inthe vector caching unit 112. The combiner may be configured to combinethe addition results to generate an output vector. The output vector maybe represented as A(1)+B(1), A(2)+B(2), . . . A(n)+B(n).

In response to a VAS instruction, the adders may be configured to add ascalar value to each element in the first vector. The output vector maybe represented as A(1)+S, A(2)+S, . . . A(n)+S.

In an example of a VS instruction, the computation module 110 mayinclude one or more subtractors configured to subtract the secondelements of the second vector from the first elements in the firstvector. The combiner may be similarly configured to combine thesubtraction results to generate an output vector. The output vector maybe represented as A(1)−B(1), A(2)−B(2), . . . A(n)−B(n).

In response to an SSV instruction, the subtractors may be configured tosubtract the scalar value from each element in the first vector. Theoutput vector may be represented as A(1)-S, A(2)-S, . . . A(n)-S.

FIG. 2B illustrates an example vector multiplication process that may beperformed by the example neural network acceleration processor.

Similarly, a first vector (“Vector A”) may include one or more elementsrespectively denoted as A(1), A(2), . . . A(n) and, similarly, a secondvector (“Vector B”) may include one or more elements respectivelydenoted as B(1), B(2), . . . B(n). The elements in the first vector maybe referred to as first elements. The elements in the second vector maybe referred to as second elements.

The computation module 110 may include one or more multipliers. Inresponse to a VMV instruction, each of the multipliers may be configuredto multiply a first element in the first vector with a correspondingsecond element in the second vector, e.g., A(1) with B(1), A(2) withB(2), . . . A(n) with B(n). The multiplication results generatedrespectively by the one or more multipliers may be directly transmittedto the combiner. In other words, the multiplication results may betransmitted to the combiner without being temporarily stored in thevector caching unit 112. The combiner may be similarly configured tocombine the multiplication results to generate an output vector. Theoutput vector may be represented as A(1)*B(1), A(2)*B(2), . . .A(n)*B(n).

In response to a VMS instruction, each of the multipliers may beconfigured to multiply a first element in the first vector with a scalarvalue. The combiner may be similarly configured to combine themultiplication results to generate an output vector. The output vectormay be represented as A(1)* S, A(2)*S, . . . A(n)* S.

In an example of a VD instruction, the computation module 110 mayinclude one or more dividers configured to divide the first element bythe second elements correspondingly. The combiner may be similarlyconfigured to combine the division results to generate an output vector.The output vector may be represented as A(1)/B(1), A(2)/B(2), . . .A(n)/B(n).

In response to an SDV instruction, the dividers may be configured todivide the first elements by a scalar value. The combiner may besimilarly configured to combine the division results to generate anoutput vector. The output vector may be represented as A(1)/S, A(2)/S, .. . A(n)/S.

FIG. 3 illustrates an example computation module in the example neuralnetwork acceleration processor by which vector operations may beimplemented in a neural network;

As depicted, the computation module 110 may include a computation unit302, a data dependency relationship determination unit 304, a neuroncaching unit 306. The computation unit 302 may further include one ormore multipliers 310, one or more adders 312, an inverter 314, areciprocal calculator 316, a combiner 318, and a vector generator 320.

The data dependency relationship determination unit 304 may beconfigured to perform data access operations (e.g., reading or writingoperations) on the caching units including the neuron caching unit 306during the computation process. The data dependency relationshipdetermination unit 304 may be configured to prevent conflicts in readingand writing of the data in the caching units. For example, the datadependency relationship determination unit 304 may be configured todetermine whether there is dependency relationship in terms of databetween a micro-instruction which to be executed and a micro-instructionbeing executed. If no dependency relationship exists, themicro-instruction may be allowed to be executed; otherwise, themicro-instruction may not be allowed to be executed until allmicro-instructions on which it depends have been executed completely.The dependency relationship may be determined when a target operationrange of the micro-instruction to be executed overlaps a targetoperation range of a micro-instruction being executed. For example, allmicro-instructions sent to the data dependency relationshipdetermination unit 304 may be stored in an instruction queue within thedata dependency relationship determination unit 304.The instructionqueue may indicate the relative priorities of the storedmicro-instructions. In the instruction queue, if the target operationrange of reading data by a reading instruction conflicts with oroverlaps the target operation range of writing data by a writinginstruction of higher priority in the front of the instruction queue,then the reading instruction may not be executed until the writinginstruction is executed.

The neuron caching unit 306 may be configured to store the elements inthe first vector and the second vector.

In some examples, the computation unit 320 may receive a scalar valuefrom the instruction caching unit 104. The vector generator 320 may beconfigured to expand the scalar value into the first vector or thesecond vector. In other words, the vector generator 320 may overwritethe elements in the first vector or the second vector with the scalarvalue. Alternatively, the vector generator 320 may generate a vector ofa same length as the first vector or the second vector. Elements of thegenerated vector may be assigned with the scalar value.

Thus, with respect to a VAS, SSV, VMS, or SDV instruction that involvesa scalar value and a vector, the vector generator 320 may convert thescalar value into a vector. The operations may be performed between avector converted from the scalar value and a received vector.

The computation unit 302 may be configured to receive themicro-instructions decoded from the vector operation instruction fromthe controller unit 106. In the example that the micro-instructionsinstruct the computation module 110 to perform a vector additionoperation to two vectors, the one or more adders 312 may be respectivelyconfigured to add a first element in the first vector to a correspondingsecond element in the second vector. The first vector and the secondvector may be retrieved from the vector caching unit 112 or may beexpanded from the scalar value from the instruction caching unit 104.

The addition results generated respectively by the one or more addersmay be directly transmitted to the combiner 318 without beingtemporarily stored in the vector caching unit 112 or the neuron cachingunit 306. The combiner 318 may be configured to combine the additionresults to generate an output vector. The output vector may berepresented as A(1)+B(1), A(2)+B(2), . . . A(n)+B(n).

In response to a VAS instruction, the adders 312 may be configured toadd a scalar value to each element in the first vector. The outputvector may be represented as A(1)+S, A(2)+S, . . . A(n)+S.

In response to a VS instruction, the subtractors 314 may be configuredto subtract the second elements of the second vector from the firstelements correspondingly. The combiner 318 may be similarly configuredto combine the addition results to generate an output vector. The outputvector may be represented as A(1)−B(1), A(2)−B(2), . . . A(n)−B(n).

In response to an SSV instruction, the subtractors 314 may be configuredto subtract the scalar value from each element in the first vector. Theoutput vector may be represented as A(1)−S, A(2)−S, . . . A(n)−S.

In the example that the micro-instructions instruct the computationmodule 110 to perform a vector multiplication operation to two vectors,each of the multipliers 310 may be configured to multiply a firstelement in the first vector with a corresponding second element in thesecond vector, e.g., A(1) with B(1), A(2) with B(2), . . . A(n) withB(n). The multiplication results generated respectively by the one ormore multipliers may be directly transmitted to the combiner 318 withoutbeing temporarily stored in the vector caching unit 112 or the neuroncaching unit 306. The combiner 318 may be similarly configured tocombine the multiplication results to generate an output vector. Theoutput vector may be represented as A(1)*B(1), A(2)*B(2), . . .A(n)*B(n).

In response to a VMS instruction, each of the multipliers 310 may beconfigured to multiply a first element in the first vector with a scalarvalue. The combiner may be similarly configured to combine themultiplication results to generate an output vector. The output vectormay be represented as A(1)*S, A(2)*S, . . . A(n)*S.

In some examples, the dividers 316 may be configured to divide the firstelements by the second elements of the second vector The combiner 318may be similarly configured to combine the division results to generatean output vector. The output vector may be represented as A(1)/B(1),A(2)/B(2), . . . A(n)/B(n).

In response to an SDV instruction, the dividers 316 may be configured todivide the first elements by a scalar value. The combiner may besimilarly configured to combine the division results to generate anoutput vector. The output vector may be represented as A(1)/S, A(2)/S, .. . A(n)/S.

FIG. 4A illustrates a flow chart of an example method 400 for performingvector addition between two vectors in a neural network. The method 400may be performed by one or more components the apparatus of FIGS. 1 and3.

At block 402, the example method 400 may include receiving, by acontroller unit, a vector addition instruction that includes a firstaddress of a first vector, a second address of a second vector, and anoperation code that indicates an operation to add the first vector andthe second vector. For example, the controller unit 106 may receive avector addition instruction that includes a first address of a firstvector, a second address of a second vector, and an operation code thatindicates an operation to add the first vector and the second vector. Afirst vector may include one or more elements respectively denoted asA(1), A(2), . . . A(n) and, similarly, a second vector may include oneor more elements respectively denoted as B(1), B(2), . . . B(n).

At block 404, the example method 400 may include receiving, by acomputation module, the first vector and the second vector in responseto the vector addition instruction based on the first address and thesecond address. For example, the computation module 110 may beconfigured to receive the first vector and the second vector in responseto the vector addition instruction.

At block 406, the example method 400 may include respectively adding, byone or more adders of the computation module, each of the first elementsto a corresponding one of the second elements to generate one or moreaddition results. For example, the one or more adders 312 may berespectively configured to add a first element in the first vector to acorresponding second element in the second vector. The first vector andthe second vector may be retrieved from the vector caching unit 112 ormay be expanded from the scalar value from the instruction caching unit104. The addition results generated respectively by the one or moreadders may be directly transmitted to the combiner 318 without beingtemporarily stored in the vector caching unit 112 or the neuron cachingunit 306.

At block 408, the example method 400 may include combining, by acombiner of the computation module, the one or more addition resultsinto an output vector. For example, the combiner 318 may be configuredto combine the addition results to generate an output vector. The outputvector may be represented as A(1)+B(1), A(2)+B(2), . . . A(n)+B(n).

FIG. 4B illustrates a flow chart of an example method 401 for performingvector addition between a vector and a scalar in a neural network. Themethod 401 may be performed by one or more components the apparatus ofFIGS. 1 and 3.

At block 452, the example method 401 may include receiving, by acontroller unit, a vector-add-scalar instruction that includes a firstaddress of a vector, a second address of a scalar value, and anoperation code that indicates an operation to add the vector to thescalar value. For example, the controller unit 106 may receive a VASinstruction that includes a first address of a vector, a second addressof a scalar value, and an operation code that indicates an operation toadd the vector to the scalar value.

At block 454, the example method 401 may include receiving, by acomputation module, the vector and the scalar value in response to thevector-add-scalar instruction based on the first address and the secondaddress. For example, the computation module 110 may be configured toreceive a first vector A and a scalar value.

At block 456, the example method 401 may include respectively adding, byone or more adders of the computation module, the scalar value to eachelement of the vector to generate one or more addition results. Forexample, the adders 312 may be configured to add a scalar value to eachelement in the first vector.

At block 458, the example. the example method 401 may include combining,by a combiner of the computation module, the one or more additionresults into an output vector. For example, the combiner 318 may beconfigured to combine the addition results into an output vector. Theoutput vector may be represented as A(1)+S, A(2)+S, . . . A(n)+S.

FIG. 5A illustrates a flow chart of an example method 500 for performingvector subtraction between two vectors in a neural network. The method500 may be performed by one or more components the apparatus of FIGS. 1and 3.

At block 502, the example method 500 may include receiving, by acontroller unit, a vector subtraction instruction that includes a firstaddress of a first vector, a second address of a second vector, and anoperation code that indicates an operation to subtract the first vectorfrom the second vector. For example, the controller unit 106 may receivea vector subtraction instruction that includes a first address of afirst vector, a second address of a second vector, and an operation codethat indicates an operation to add the first vector and the secondvector. A first vector may include one or more elements respectivelydenoted as A(1), A(2), . . . A(n) and, similarly, a second vector mayinclude one or more elements respectively denoted as B(1), B(2), . . .B(n).

At block 504, the example method 500 may include receiving, by acomputation module, the first vector and the second vector in responseto the vector subtraction instruction based on the first address and thesecond address. For example, the computation module 110 may beconfigured to receive the first vector and the second vector in responseto the vector subtraction instruction.

At block 506, the example method 500 may include respectivelysubtracting, by one or more subtractors of the computation module, eachof the first elements from a corresponding one of the second elements togenerate one or more subtraction results. For example, the subtractors314 may be configured to subtract the second elements of the secondvector from the first elements correspondingly.

At block 508, the example method 500 may include combining, by acombiner, the one or more subtraction results into an output vector. Forexample, the combiner 318 may be similarly configured to combine thesubtraction results to generate an output vector. The output vector maybe represented as A(1)−B(1), A(2)−B(2), . . . A(n)−B(n).

FIG. 5B illustrates a flow chart of an example method 501 for performingvector subtraction between a vector and a scalar value in a neuralnetwork.

At block 552, the example method 501 may include receiving, by acontroller unit, a scalar-subtract-vector instruction that includes afirst address of a vector, a second address of a scalar value, and anoperation code that indicates an operation to subtract the vector fromthe scalar value. For example, the controller unit 106 may receive asSSV instruction that includes a first address of a vector, a secondaddress of a scalar value, and an operation code that indicates anoperation to subtract the vector from the scalar value.

At block 554, the example method 501 may include receiving, by acomputation module, the vector and the scalar value in response to thescalar-subtract-vector instruction based on the first address and thesecond address, wherein the vector includes one or more elements. Forexample, the computation module 110 may be configured to receive a firstvector A and a scalar value.

At block 556, the example method 501 may include respectivelysubtracting, by one or more subtractors of the computation module, thescalar value from each element of the vector to generate one or moresubtraction results. For example, the subtractors 314 may be configuredto subtract the scalar value from each element in the first vector.

At block 558, the example method 501 may include combining, by acombiner, the one or more subtraction results into an output vector. Forexample, the combiner 318 may be similarly configured to combine thesubtraction results to generate an output vector.

The process or method described in the above accompanying figures can beperformed by process logic including hardware (for example, circuit,specific logic etc.), firmware, software (for example, a software beingexternalized in non-transitory computer-readable medium), or thecombination of the above two. Although the process or method isdescribed above in a certain order, it should be understood that someoperations described may also be performed in different orders. Inaddition, some operations may be executed concurrently rather than inorder.

In the above description, each embodiment of the present disclosure isillustrated with reference to certain illustrative embodiments.Apparently, various modifications may be made to each embodiment withoutgoing beyond the wider spirit and scope of the present disclosurepresented by the affiliated claims. Correspondingly, the description andaccompanying figures should be understood as illustration only ratherthan limitation. It is understood that the specific order or hierarchyof steps in the processes disclosed is an illustration of exemplaryapproaches. Based upon design preferences, it is understood that thespecific order or hierarchy of steps in the processes may be rearranged.Further, some steps may be combined or omitted. The accompanying methodclaims present elements of the various steps in a sample order, and arenot meant to be limited to the specific order or hierarchy presented.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein but is to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. All structural andfunctional equivalents to the elements of the various aspects describedherein that are known or later come to be known to those of ordinaryskill in the art are expressly incorporated herein by reference and areintended to be encompassed by the claims. Moreover, nothing disclosedherein is intended to be dedicated to the public regardless of whethersuch disclosure is explicitly recited in the claims. No claim element isto be construed as a means plus function unless the element is expresslyrecited using the phrase “means for.”

Moreover, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom the context, the phrase “X employs A or B” is intended to mean anyof the natural inclusive permutations. That is, the phrase “X employs Aor B” is satisfied by any of the following instances: X employs A; Xemploys B; or X employs both A and B. In addition, the articles “a” and“an” as used in this application and the appended claims shouldgenerally be construed to mean “one or more” unless specified otherwiseor clear from the context to be directed to a singular form.

We claim:
 1. An apparatus for vector operations in a neural network,comprising: a controller unit configured to receive a vector additioninstruction that indicates a first address of a first vector, a secondaddress of a second vector, and an operation code that indicates anoperation to add the first vector and the second vector; and acomputation module configured to receive the first vector and the secondvector in response to the vector addition instruction based on the firstaddress and the second address, wherein the first vector includes one ormore first elements and the second vector includes one or more secondelements, and wherein the computation module includes: one or moreadders configured to respectively add each of the first elements to acorresponding one of the second elements to generate one or moreaddition results, and a combiner configured to combine the one or moreaddition results into an output vector.
 2. The apparatus of claim 1,wherein the vector addition instruction further indicates a first lengthof the first vector, and wherein the computation module is configured toretrieve the first vector based on the first address and the firstlength.
 3. The apparatus of claim 1, wherein the vector additioninstruction further indicates a second length of the second vector, andwherein the computation module is configured to retrieve the secondvector based on the second address and the second length.
 4. Theapparatus of claim 1, wherein the vector addition instruction furtherincludes one or more register IDs that identify one or more registersconfigured to store the first address of the first vector, a firstlength of the first vector, the second address of the second vector, anda second length of the second vector.
 5. The apparatus of claim 1,wherein the controller unit comprises an instruction obtaining moduleconfigured to obtain the vector addition instruction from an instructionstorage device.
 6. The apparatus of claim 5, wherein the controller unitfurther comprises a decoding module configured to decode the vectoraddition instruction into one or more micro-instructions.
 7. Theapparatus of claim 6, wherein the controller unit further comprises aninstruction queue module configured to temporarily store the vectoraddition instruction and one or more previously received instructions,and retrieve information corresponding to operation fields in the vectoraddition instruction.
 8. The apparatus of claim 7, wherein thecontroller unit further comprises an instruction register configured tostore the information corresponding to the operation fields in thevector addition instruction.
 9. The apparatus of claim 8, wherein thecontroller unit further comprises a dependency processing unitconfigured to determine whether the vector addition instruction has adependency relationship with the one or more previously receivedinstructions.
 10. The apparatus of claim 9, wherein the controller unitfurther comprises a storage queue module configured to store the vectoraddition instruction while the dependency processing unit is determiningwhether the vector addition instruction has the dependency relationshipwith the one or more previously received instructions.
 11. A method forvector operations in a neural network, comprising: receiving, by acontroller unit, a vector addition instruction that includes a firstaddress of a first vector, a second address of a second vector, and anoperation code that indicates an operation to add the first vector andthe second vector; receiving, by a computation module, the first vectorand the second vector in response to the vector addition instructionbased on the first address and the second address, wherein the firstvector includes one or more first elements and the second vectorincludes one or more second elements; respectively adding, by one ormore adders of the computation module, each of the first elements to acorresponding one of the second elements to generate one or moreaddition results; combining, by a combiner of the computation module,the one or more addition results into an output vector.
 12. The methodof claim 11, further comprising obtaining, by an instruction obtainingmodule of the controller unit, the vector addition instruction from aninstruction storage device.
 13. The method of claim 12, furthercomprising decoding, by a decoding module of the controller unit, thevector addition instruction into one or more micro-instructions.
 14. Themethod of claim 13, further comprising temporarily storing, by aninstruction queue module of the controller unit, the vector additioninstruction and one or more previously received instructions, andretrieve information corresponding to operation fields in the vectoraddition instruction.
 15. The method of claim 14, further comprisingstoring, by an instruction register of the controller unit, theinformation corresponding to the operation fields in the vector additioninstruction.
 16. The method of claim 15, further comprising determining,by a dependency processing unit of the controller unit, whether thevector addition instruction has a dependency relationship with the oneor more previously received instructions.
 17. The method of claim 16,further comprising storing, by a storage queue module of the controllerunit, the vector addition instruction while the dependency processingunit is determining whether the vector addition instruction has thedependency relationship with the one or more previously receivedinstructions.